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Abstract 

For estimating the unknown parameters in an unstable autoregres- 
sive AR(p), the paper proposes sequential least squares estimates with 
a special stopping time defined by the trace of the observed Fisher in- 
formation matrix. The limiting distribution of the sequential LSE is 
shown to be normal for the parameter vector lying both inside the 
stability region and on some part of its boundary in contrast to the 
ordinary LSE. The asymptotic normality of the sequential LSE is pro- 
vided by a new property of the observed Fisher information matrix 
which holds both inside the stability region of AR(p) process and on 
the part of its boundary. The asymptotic distribution of the stopping 
time is derived. 

AMS 1991 Subject Classification : 62L10, 62L12. 

Key words and phrases: Autoregressive process, least squares estimate, 
sequential estimation, uniform asymptotic normality. 



*This research was carried out at the Department of Mathematics, the Strasbourg 
University, France. The second author is partially supported by the RFFI-DFG-Grant 
02-01-04001. 



1 



1 Introduction 

Consider the autoregressive AR{p) model 

Xfi OiXfi^i -|- . . . -|- OpXn—p ~\~ £ni ^ l)^,..., (1.1) 

where (xn) is the observation, (e„) is the noise which is a sequence of inde- 
pendent identically distributed (i.i.d.) random variables with E^i = and 
< Eel = cr^ < oo, 0"^ is known, xq = x_i = . . . = Xi-p = 0; parameters 
of the model 6i, . . . ,6p are unknown. This model can be expressed in vector 
form as 

X„ = AX„_i+a, (1.2) 
where X„ = (x„, x„_i, . . . , Xn-p+i)', ^ = {Sn, 0, . . . , 0)', 

^ - ( ) • (-) 

the prime denotes the transposition. 

A commonly used estimate of the parameter vector 9 = (6*1, ... , Op)' is 
the least squares estimate (LSE) 

n n 

e{n) = M-'J2^k-iXk, M„ = J2Xk-iX',^i, (1.4) 

fc=l k=l 

where denotes the inverse of matrix M„ if det M„ > and = 
otherwise. Let 

V{z) = zP- OizP-^ -...-Op (1.5) 

denote the characteristic polynomial of the autoregressive model fll.l I) . The 
process f ll.l p is said to be stable (asymptotically stationary) if all roots 
Zi = Zi{6) of the characteristic polynomial (11.5 I) lie inside the unit circle, that 
is the parameter vector 9 = {9i, . . . , Op)' belongs to the parametric stability 
region Ap defined as 

Ap = {eGi?^:|z,(^^)|<l,z = l,...,p}. (1.6) 

The process f ll.l \\ is called unstable if the roots of V{z) lie on or inside the 
unit circle, that is, 9 G [Ap]; [Ap] denotes the closure of the stability region 
Ap. 

It is well known (see,e.g. Anderson (1971), Th.5.5.7) that the LSE 9{n) 
is asymptotically normal for all 9 & Ap, that is 

as n — > oo, 
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where F = F(9) is a. positive definite matrix, =^ indicates convergence in 
law. It should be noted that the asymptotic normality of 9{n) is provided by 
the following asymptotic property of the observed Fisher information matrix 

lim M„/n = F a.s. (1.7) 

n—*oo 

for all 9 e Ap. On the boundary dAp of the stability region Aj,, this property 
does not hold and the distribution of 9{n) is no longer asymptotically normal. 
The investigation of the asymptotic distribution of LSE 9{n) when Xn is 
unstable goes back to the late fifties with the paper of White (1958) (see 
also Ahtola and Tiao (1987), Dickey and Fuller (1979), Rao (1978), Sriram 
(1987), (1988)) who considered the AR(1) model with i.i.d. ^^(O,^^) random 
errors £„ and 9i — 1 and estabhshed that 

n{9{n) - 1) ^ {W\l) - 1)/ f W\t)dt, 

Jo 

where W{t) is a standard brownian motion. Subsequently the research of the 
limiting distribution of 9{n) for unstable AR(p) processes has been receiving 
considerable attention due to important applications in time series analysis, 
in modeling economic and financial data and in system identification and 
control. We can not go into the detail here and refer the reader to the paper 
by Chan and Wei (1988) who derived the limiting distribution of LSE 9{n) for 
the general unstable AR(p) model. By making use of the functional central 
limit theorem approach, Chan and Wei expressed the limiting distribution 
of LSE 9{n) in terms of functionals of standard brownian motions. However, 
the closed forms of the distribution functions of these functionals are not 
known and that may cause difficulties in practice (see section 4 in Chan and 
Wei). 

For the unstable AR(1) model with i.i.d. random errors and — 1 < < 
1, Lai and Siegmund (1983) proposed to use the sequential least squares 
estimate for 9i which is obtained from the LSE 

n n 
fc=l k=l 

by replacing n with a special stopping time r based on the observed Fisher 
information. They proved that, in contrast with the ordinary LSE 9i{n), the 
sequential LSE is asymptotically normal uniformly in 9 e [—1,1]- For the 

unstable AR(2) model, Galtchouk and Konev (2006) applied the sequential 
LSE with a particular stopping time and established that it is asymptotically 
normal not only inside the stability rigion A but also for its boundary points 
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6 corresponding to a pair of conjugate complex roots Zi = e*"^, Z2 = e"*"^ of 
the polynomial fll.5 I) . 

In this paper, for the case of unstable AR(p) process, we propose a se- 
quential LSE for 6 and find the conditions on 6 (see Conditions 1-3 in the 
next section) ensuring its asymptotic normality. The set Ap of the points 6 , 
satisfying these conditions includes the stability region Ap and some part of 
its boundary. It is shown that the convergence of the sequential LSE to the 
normal distribution is uniform in 6 E K for any compact set K E Ap (see 
Theorem 2.1). The extension of the property of asymptotic normality of the 
sequential estimate to the part of the boundary dAp is achieved by making 
use of a new property of observed Fisher information matrix M„, which holds 
in a broader subset of [Ap] as compared with fll.7 I) (see Lemma 3.3). 

The remainder of this paper is arranged as follows. In Section 2 we 
introduce a sequential procedure for estimating the parameter vector 6 = 
{9i, . . . , Op)' in (11. 1 I) and study its properties. Section 3 gives a new property 
of the observed Fisher information matrix and establishes some technical 
results. In Section 4 we prove Theorem 2.2 from Section 2 on asymptotic 
distribution of the stopping time. 

2 Sequential least squares estimate. 
Uniform asymptotic normality. 

In this section we consider the sequential least squares estimate and study 
its asymptotic properties. We define the sequential LSE for the parameter 
vector 6* = (6*1, ... , Op)' in model (11.1 I) as 

T{h) 

0(r(/i)) = M;(i)^X,_iXfc, (2.1) 

A;=l 

where 

T{h) = inf {n > 1 : tr M„ > /la^} , inf{0} = +00, (2.2) 

is stopping time, /i is a positive number (threshold). 

Assume that the parameter vector = (6^1, ... , Op)' in (11.1 p satisfies the 
following Conditions. 

Condition 1. Parameter = (^^i, . . . , Op)' is such that all roots Zi = Zi{0) 
of the characteristic polynomial (11.5 p lie inside or on the unite circle. 

Condition 2. All the roots Zi = Zi{0) of V{z), which are equal to one in 
modulus, are simple. 
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Condition 3. The system of linear equations witli respect to Yi, . . . , 



2<j<p-l, 



(2.3) 



has a unique solution (Yi, . . . , Yp_i), = K,i{6),l < i < p — 1, and the matrix 



L{9i, ■ ■ ■ ,9p 



( 1 K^{e) K2{e) 



Kp.,{e) \ 

t^p-2{0) 
1 J 



(2.4) 



is positive definite. 

o 

Let Ap denote all = (^^i, . . . , Op)' in f ll.l I) which satisfy Conditions 1,2, 
and Ap- all 9 = {6i, . . . , Op)' satisfying all Conditions 1-3. 
Example 2.1. For AR(2) process, one finds 



^2 = {0= {01, 02)' : -1 + 02 < ^1 < 1 - ^2, 1^21 < 1} , 
[A2] = {0 = {01, 02)' : -1 + ^^2 < ^^1 < 1 - ^^2, \02\ < 1} , 

Ap=[A2]\ {(-2,-1), (2,-1)}, 

A2 = {0= {01, 02)' : -1 + 02 < 01 < 1 - 02, -1 < 02 < 1} 

r(f) 1 0l/(l-02) 

^y^'^^^>-[9^/{l-02) 1 

Example 2.2. By numerical calculation for AR(3) process, one can check that 
Conditions 1-3 are satisfied, for example, for the values of = (0i, 02, 03) such 
that zi{0) = e'^, Z2{0) = e''^ with 37r/10 < (f) < 3tt/5 and -1 < ^3(0) < 
-0.5. 

As is shown in Lemma 3.3 (Section 3), Conditions 1-3, imposed on the 
parameter = (0i,...,0p)' in fll.l p . provide the convergence of the ratio 
^n/Ylk=i ^k-1 matrix L{0i, . . . ,0p) given in (12.4 1) . This property 

can be viewed as an extension of (11.7 I) outside the stability region (11.6 I) . 

Remark 2.1 It will be observed that Ap C Ap and, for all G Ap, one has 

Mn pF 



™Efc=ia;;_i trF 



A(0i, . . . , 0p) a.s. 



(2.5) 
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where F is the same as in ( [i. 7 [ ). Indeed, by making use of the identity 

E^ti = ^E 11^^-11' + iE E ^^-1' (2-6) 



k=l 



P 



k=l 



P 



1=2 l=n-i+2 



one obtains 



p n 



k=l i=2 l=n-i+2 



En 2 I 

fc=i a^Li n \j) n 

Limiting n — » oo, one comes to Ii2.5 \) . in view of ( [i. 7 [ ). 



Theorem 2.1 Suppose that in the AR(p) model U.l \) . is a sequence of 
i.i.d. random variables with = and Ee^ = a"^ < oo and the parameter 
vector 6 = {6i, . . . ^Op)' satisfies Conditions 1-3. Then for any compact set 

K cK 



lim sup sup 



{MHl^io{T{h))-e)<t)-%{*- 



0, 



(2.7) 



where = ■ • • $(tp), $ is the standard normal distribution function, 
Ap is defined in Condition 3. 



Proof. Substituting (ITTD in (12X1) yields 

Tih) 



Ml[l^{d{T{h)-9) = M;^;^/' = ^rhM;l^:^L^'\9^,...,^p)Yu, (2.8) 
where 



k=l 



^ r{h) 

Yh = —7= / ] L^^^'^i^i, • • • , 9p)Xk_iek 
and L{6i, . . . ,9p) is given in 02.4 I) . Denote 

Grih) = L-'^\Oi, ep)Mr^h)L-'/\e^, ...,ep 

One can easily verify that 



(2.9) 



M 



1/2 



L ^/^(^i, . . .,6p) ^ - Ip 



< Wh-'Grih) - Iplr < trL-'ie,, . . . , ep)\\h-'M,^h) - 1(6,, . . . , a 



From here, by making use of Lemma 3.4 from Section 3, one gets, for any 
compact set K G Ap and 6 > 0, 

lim sup Pe (\\VhM-];.^L'/\e^, . . . ,6^) - IJ > 6) = . (2.10) 

Now we prove that for any compact set K G Ap and for each constant vector 
V E BP with ||f II = 1 

lim sup sup \Pe{v'Yh < t) - $(t)| =0. (2.11) 
In view of fl2.9 I) , one has 

^ r(h) 

For each > 0, we define an auxihary stopping time as 

n 

T-0 = ro{h) = inf{n > 1 : ^Li > h}, inf{0} = +oo. 

k=l 

Further we make use of the representation 

v'Yh = ^y\ Qk-iSk + vW + A{h) , 
where A{h) = Mh) , 

Az{h) = — /i""^/^/(^Q(ft)=i)5(o£:i, A4(/;,) = h^^^'^gn,{h)-i£To(h) , 

^ r{h)-l ro(h)-l 

Vih) = ^ dk-iSk --7t Y I{ro{h)>i)9k^i£k ■ 

k=l k=l 

Now we show that 

lim sup sup \Pe{^ V gk-i£k < t) - ^t)\ = (2.12) 

k=l 



'eeK teR 

and for any 6 > 



hm supPe(|r/(/i)| > (5) = 0, (2.13) 



eeK 



lim supPe(|A(/i)| > 5) = 0. (2.14) 

The proof of (12.12 p is based on Proposition 3.1 from the paper by Lai and 
Siegmund (1983). Actually one needs to check only the condition Aq, that 
is, for each 5 > 0, 



lim supPe((7^ — / 9k-i some n > m) = . (2-15) 
Conditions Ai — Ar, are evidently satisfied. It will be noted that 

n / n \ n 

k=l \ k=l J k=l 

Proceeding from this equality one gets the inclusion 



{dn ^ ^ E] 9k~i some n > m} C 

k=l 

n 

C {||X„||^ > Si E^ gl_i for some n > m} 
k=i 

n n 

= {\\Xnf >SiYl xl-i[^+v'L-^''^{Mn/ xl_^-L)L-^l'^v] for some n>m} 

k=l k=l 
n n 

C {||X„f > (5iE x^_Jl - WL^'^vW^WMn/Y^ xl_^-L\\] for some n>m} 

k=l k=l 

n n 

C {||X„f > 4_i[l -a*||M„/E xl_^-L\\] for some n > m} 

k=l k=l 

n 

C {||M„,/E xl_^ - L|| > (2a*)"^ for some n > m} 

U{||X„||^ > — E^ a;^_i for some > m} , 

where b\ = 6/a*,a* = sup^g^ 

This, in view of Lemmas 3.1,3.3, yields (12.15 p . It will be observed that 
(12.15 p enables one to show (by the same argument as in Lemma 3.5) that, 
for any compact set K (Z Ap and 6 > 



ro-l 

hm svipPeigl^jJ2 9l-i>^)=^- (2-16) 



k=l 
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Now we check ( 12.13 I) . One can easily verify that 

, -r-1 



Eev\h)=Eeu{h),u{h) = - 

n 



TQ-l 



5Z - 



k=l 



k=l 



The random variable u{h) is uniformly bounded from above uniformly in 
9 E K because 

k=l 

. r-1 



^lE \\Xk-ir + l<a* + l. 



k=l 



Therefore, it suffices to show that for each 6 > 

lim sup Pg{u{h) > 6) = 

h->oo 0(zK 

To this end, one can use the following estimate 



(2.17) 



TQ-l 



k=l 

V k=l 



r-1 



k=l 



-1/2, 



k=l k=l 
T{h)-1 -r(/i)-l To{h)-l 

<a*\\Mrih)-l/ Yl 4-1 - L\\ + 4{h)-l/ Y 4-1 + 9ro{h)-l/ Y 3k-i- 
k=l k=l k=l 

From here, by making use of (12.16 \\ and Lemmas 3.4,3.5 one comes to (12.17 \\ 
which implies (12.13 I) . By a similar argument, one can check (12.14 I) . This 
completes the proof of (12.11 I) . Combining (12.10 I) and (12.11 p one arrives at 
(I^TD . Hence Theorem 2.1. □ 

Now we will study the properties of the stopping time r(/i) defined by 
(12.2 I) . Further we need the following functionals 



Ji{x;t) = x'^{s)ds, 

J2(x, y; t) = /q {x\s) + y\s))ds, 

y; t) = {x^{s) + fiiy^{s))ds, 
Ji{x, y, z; t) = J2{x, y; t) + fi2Ji{z; t), 
J5(x, y, z, u; t) = J2(x, y; t) + n^M^', t) + fJ-iMu; t), 



(2.18) 
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where /x^, z = 1, 4 are defined by KTT} . KTn and i^M}- For the set Ap of 
the parameter vector 9 = {9i, . . . , Op)' satisfying Conditions 1-3, we introduce 
the following subsets belonging to its boundary dAp 

Ti{p) = {ee dAp : zi{e) = -1, \z,{e)\ < l,k = 2^} 

r2(p) = {0e dAp : z,{e) = 1, \zk{e)\ <i,k = 2,p} 

Tsip) = {ee dAp : z,{e) = e''f>,Z2{e) = G (0,7r), \zk{e)\ <l,k = 3,p} 

T^{p) = {ee dAp : zi{e) = -i,z2{e) = i, \zk{e)\ <i,k = 3,p} 

r,{p) = {de dAp^ziid) = -l,Z2{e) = e'^,Z3{e) = e-^\<Pe (0,7r), 
\zk{e)\ <l,k = 4,p}, 

T^{p) = {ee dAp^zi{e) = l,Z2{e) = e^^^sW = G (0,7r), 

\zk{e)\ < i,k = 4,p} 

Trip) = e dAp^z.ie) = -l,Z2{e) = l,zs{e) = e"t>,z,{e) = G (0,7r) 

\zk{0)\ <l,k = 5,p}, 

(2.19) 

where Zk{0) are roots of the characteristic polynomial (11. 5 p . 

It will be noted that all these sets will be used only for the AR(p) model 
(11.1 I) with p > 5. In the case when p < 4, it is obious which of the sets Ti{p) 
are odd and how to amend the remaining subsets Ti{p). 

Theorem 2.2 Suppose that in the AR(p) model U.l \) . (£„)„>! is a sequence 
of i.i.d. random variables with E£„ = 0, < Ee^ = cr^ < oo and the 
parameter vector 6 = {6i, . . . ,6p) satisfies Conditions 1-3. Let T{h) be defined 
by ^2.2 \) . Then, for each 9 G Ap, 

P, - lim ^ = --. 2.20 

h^oo h trF 

Moreover, for each 9 G dAp, as h ^ oo, 

p^^r,^^ if^Gr,(p),l<z<7, (2.21) 

where Ap is given in lil.6 \) : 

ui = inf {t > : Ji{Wi]t) > 1}, 

z/2 = inf{t>0: Ji(Vr2;t)>l}, 

1/3 = inf {t > : J2{Wi, W^a; t) > 1} , 

1/4 = inf {t > : J^iWi, W2] t) > 1} , 

Vi = inf {t > : J4(W^i, W^2, W^- 1) > 1} , z = 5, 6, 

vr = inf {t > : 75(^^1, W^2, W^s, W^-t)>l}- 

hi,..., h are defined by (fTT]), (fCT, UJT]) , gTT]) and gM^, re- 
spectively; Wi{t), . . . , W^lt) are independent standard brownian motions. 

The proof of Theorem 2.2 is given in the Appendix. 
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3 Auxiliary propositions. 

In this Section we establish some properties of the process (11 .1 \\ and the 
observed Fisher information matrix M„ used in Section 2. 

We need some notations. Let zi{9) , . . . , Zg{9) denote all the distinct 
roots of the characteristic polynomial (11.5 I) . mi{6) be the multiplicity of 
(ELi'^.W=P)- Let 



Pid) 



max(mj(6') : = 1) if max|zj(0)| = 1, 

if max \zi{9)\ ^ 1 for all i = 1, g. 



Formally the set Ap introduced in Condition 3 can be written as 

Ap = Ap U : max \zi{e)\ = 1, p{e) = 1}. (3.1) 

l<i<q 

It includes both the stability region Ap and the points 6 of its boundary 
for which all the roots of the polynomial (11.5 I) , lying on the unit circle, are 
simple. 

Lemma 3.1 Let {xn)n>o be an autoregressive process defined by 1^1.1 I) . Then 

o 

for any compact set K C Ap and 6 > 



lim sup P6)( max > ^ / ^l-i ^'^^ some n > m) = . 

fc=l 

Proof. Taking into account the equation (11.2 I) , it suffices to show that for 

o 

any compact set K C Ap and 6 > 

lim supP,(S„(5)) = 0, 

m^oo 

where 

n 

-Bm(^) = {ll-'^rilP ^ '^X^ ll-'^fc-ilP foi" some n > m}. 

k=l 

Now we estimate the ratio X]fc=i ll^fc-iP from above. For each 1 < 

s < n, we introduce the quantity 

L = minjl <i<s: min ||Xt.„,-p = 
i<i<s 

and have the inequality 

n s 

J2 ll^^-ill' > E ll^-'^ll' ^ ^Xn^isf- (3.2) 

k=l k=l 
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On the other hand, it follows from fll.l p that 

h-i 

and, therefore, one gets 

||X„f < 2p'^f ||X„_^Jp + 2||^A^a- 



^=o 



Is-l 



1=0 



' s-1 



s-1 



. i=0 



i=l 



s-1 



(3.3) 



Further it will be observed that, for every compact set K C Ap, there exists 
a positive number k such that 



sup||A"||^ < K, n>l. 



(3.4) 



Indeed, we express A in its Jordan normal form 

A = SVS-^ , (3.5) 
where V = diag(Ji, . . . , Jq), Ji is the mi x m; submatrix of the form 



Ji 



( zi 1 ... \ 

1 ... 

... zi \ 

\ ... ) 



if 2;; is a multiple root with multiplicity mi > 2, and J; = 2;; if zi is a simple 
root. 

By direct computation with (13.5 I) one finds A" = SV^S~^, = diag( J", . 
where the powers of the matrix Ji are equal to for a simple root zi and 
consist of the elements (see, R.Varga (2000)) 



< JP >ij- 



0, J <i, 

J — z) z^~-^~^\ < J < min(mi, n + z) 
n + i < j < m; , 



0, 



12 



for the roots Zi with multiphcity mi > 2, — is the binomial coefficient. 

From here, in view of the definition fl3.1 I) , one comes to (13.4 I) . By making 
use of (13.3 I) and fl3.4 p . one obtains 



s-l 



|X„f < 2/€||X„_zJ|2 + 2sfi:^ 



i=0 



Combining this inequahty and (13.2 p yields 



I Y l|2 Ouc 2 



It remains to use elementary inequality 



n— 1 n 

5^4<2(l+Pf)5^||X,_if , 

i=0 k=\ 

which follows from fll.2 P . to derive the desired estimate for the ratio 



This inequality implies the inclusion 

Bm{6) C {2k/ s > 6/2 for some n > m} 

S—l 71—1 ^ 

U{4(1 + ||A|HsK^£:-_i/^4 > 2 n>m}. 

i=0 k=l 

Therefore, for sufficiently large s, one gets 

s—l n— 1 ^ 

snpPe{Bm{6) < P{i^/ ^l-il / ^1 > ~ some n > m} , 

tit 2 

where z/ = supgg^4(l + ||y4|p)s/€. Limiting m oo and applying the law of 
large numbers one comes to the assertion of Lemma 3.1. 

Lemma 3.2 Let {xn)n>o be an autoregressive process defined by 1^1.1 I) . Then 

o 

for any compact set K C Ap and each I = l,p — 1 



lim supPe( 



k=l 



— ^ -^k-i some n > m) = , (3.6) 



k=l 



where Ap is given in ^3.1 ]) 
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Proof. We will apply Lemma 2.2 from the paper by Lai and Siegmund (1983)) 
given in the Appendix. Let c„ = n^/^. For the set of interest one has the 
following inclusions 



Xk-iSk\ > S x1_i for some n> i 

k=l k=l 

n n 

— I I ^k-i£k\ > ^ for some n > m 

> 5 for some n > m 



k=l k=l 



^ I T^^^i^WV^ > ^ for some n > ml U 

n n 

C^xl_i)~'^/^ V Cn(^a;^_;)"^ > for some n > m 

k=l k=l 



C < I ^ Xk-iSkl ^l-iT^^ V c„ j > for some n > m > U 

k=l \ k=l J 

U < a;|-z)~^ > (^^^^ for some n > m > U 
I jfc=i 

r " 

U < c„(^ ^l-i) ^ > for some n > 
I fc=i 

n n 

C < I ^ a;fc_i£jk| > (^l^ V c„ for some n > 



k=l 

n 

— ^ 

fc=l 

n n 

A;=l A;=l 

n 



u 



some n > m > . 

k=l 

Prom here, it follows that 

(n n 
I Xk-iSk\ > S for some n> m 

k=l k=l 

/ n / n \ 2/3 

< I I Xk-iSk\ > ^ I xl_i j V c„ for some n> m 



k=l \k=l 
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+ Pe \ n^^^Yl ^l-i^^ >V5A 5^'^ for some n>m\. (3.7) 



k=l 



By making use of (11. 1 p and the elementary inequalities, one obtains 

n n 
^ 4 = ^{^k - OiXk-i - ... - 9pXk- 



-p) 

k=l k=l 



p n 

2 

k~j 

k=l j=l k=l 



<(p+i) + 



n p n 

< (p + 1) rn^/KE + E E ^t.) 

k=l j=l k=l 

n / n n 

A;=l \ A;=n-i+l A:=l 

where /x^ = supgg^ II^IP- 

Therefore the second summand in the right-hand side of (13.6 I) can be 
estimated as 



Pe < n^^^Yl ^l-iy^ > A 5^/2 for some n > 

I k=l 

< P J 2{p + lffikn^/\J2 ^Ir^ >V6A S^/^ for some n > m 

I k=l 

{n n 
E/ *^^/ E/ "^^-^ — some n > m 
k=n-l+l k=l 

Combining this and (13.7 I) and applying Lemma 3.1, and Lemma 2.2 by Lai 
and Siegmund (see Section 4) yield (13.6 p . 

This completes the proof of Lemma 3.2. □ 

Lemma 3.3 Let parameters 6i, . . . ,6p in the equation lil.l \) satisfy Condi- 
tions 1-3 and p x p matrix Mn be given by lil.3 I) . Then, for any compact set 
K G Ap and each 6 > 0, 

lim sup Pg ( II — % L{9i, ■ ■ ■ ,9p)\\ > 6 for some n > m] = 0, 



where L{6i, . . . ,6p) is defined in 1^2.4 \j 
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Proof. Each diagonal element of the matrix M„ can be expressed through 

En 2 
1=1 ^l-l 



n—i+1 



k=i 1=1 1=1 l=n-i+2 

Further it will be observed that each element < M„ >ij, 2 < i < j < p, 
of Mn standing above the principal diadonal and below the first row can be 
expressed through some element of the first row as 

n n n 

< Mn >ij= 

•^k—i-^k—j / j -^l—l-^l—l+i—j / J •^l—l-^l—l+i—j 

(3.9) 

k=l 1=1 l=n-i+2 

n 

= < Mn >ij_i+l — Xt-l+i-j . 

t=n-i+2 

Now we derive the equations relating the elements of the first row, that is, 
< Mn >is, 2 < s < p. Making use of the equation (11.1 p . one gets 

n n / p \ 

< Mn >ls= Yj ^k-lXk-s = Yj\11 ^l^k-l-1 + ^k-1 j Xk-s 
k=l k=l \l=l / 

p n n 

= Y^^'Y ^k-l-lXk~s + ^ Xk-s£k-l ■ 
1=1 k=l k=l 

Since 

n ( < Mn >ss, if / = S - 1, 

Y^k-l-lXk-s = I <Mn>s,l+l, if / > S, 
k=l ( < Mn >l+l,s, if / < s - 2, 

this implies the following system of equations 

p n 
< Mn >12= 01 < Mn >22 + ^ < M„ >2,l+l + ^ Xfe_2^fe-1 , 

1=2 k=l 



< Mn >ls= ^ < M„ +es-l < Mn >ss 

1=2 

p n 
+ < M„ >s,l+l + Y ^k-s^k-1, 3 < S < p. 



k=l 
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Taking into account (13.8 I) . (13.9 I) one can represent this system as 

n p 

< M„ >i2= OiY,4-i + < Mn >i,i +Vi,2{n), (3.10) 



k=l 1=2 
s—2 n 



< Mn >ls=^Ol < Mn >l,s-l +es-l^xl^^ + ^ei < M„ >l,l-s+2 +^?M(n), 
1=1 k=l l=s 

where 3 < s < j9, 

p n 

1=2 k=l 
s—2 n n 

VlA'^) = -"^^l Xt-lXt+l-s - ds-l ^ Xf_^ 

1=1 t=n~l+l l=n-s+2 
p n n 

^ Xt^lXt+s-l-2 + y^^Xk-s£k-l- 



l=s t=n-s+2 k=l 



Denote 



_ < Mn >i,i+i ^ _ rji^i+i{n) . _ _ 

Z^fc=l-^fc-l 2^fc=l^fc-l 



Then the system of equations (13.10 p takes the form 

- Zir=2 OlZl-l{n) = di+ 7^1,2, 

- Efc=l^i-fc ^fc(n) + ^j(^) - ECi dk+jZk{n) = 9j + Vij+i, (3.11) 
J=2,p-1 

In virtue of Lemmas 3.1,3.2, for any compact set K G Ap and 6 > 0, one has 
hm supP6i(max |r/i,s(Ti)| > 6 for some n>m)=0. 

m— >cxD 2<s<p ' 

From here and the Condition 3, which holds for each vector ^ G i^, it follows 
that the solution of the system (13.11 h converges, as cxd, to the unique 
solution of system fl2.3 I) uniformly in 6* G -ft', that is, 

lim supP6)( max \zi{n) — Kj(^)| > 6 for some n > m) = 0. 

m^oo l<i<p—l 

This, in view of (13.9 I) and Lemma 3.1, implies the desired convergence of the 
remaining elements of the matrix M„. Hence Lemma 3.3. □ 
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Lemma 3.4 Let M„, T{h) and L{9) = L{9i, ...,9p) be given by ^1.4 \) , \2.2 \) 

o 

and ^2.4 \ ), respectively. Then, for any compact set K C Ap and 6 > 0, 

lim supP, (\\^^ - L{9i, ...,9p)\\>6] =0, (3.12) 

o 

where Ap is defined in Condition 3. 
Proof. By making use of the equality 

^ - m = - m + M.(.) ( 1 - --^ 

one gets the estimate 

\\'^-Lm\<\\-^-Lm 

/ rih) \ /rih) \ 



k=l / \ k=l 



Therefore 



'^^^'^ m\\>4c\\\-^^-Lm>S/2} (3.13) 



L Z^k=l -^k-l 

. , Mr(h) (Y.l=i^l-i-h 



Further one has the inclusions 



J^f!, - m\\ >5/2)Cl {r{h) < m} 

l^k=l ^k~l 

U ^ II ^^^H — ~"2 -^(^)ll > for some n > m 

l^k=l ^k-l 



J24-i-h\ /h>5/2\ C{rih)<m} 



k=l 
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1 Mn 

U <^ II ™ 2 -^(^)ll > for some n>m 



r{h) 

J24^i-h] /h>6/2\ C{r{h)<m} 



k=l 



U { ■ ^n-"i '2 > some n > m }> . 

From here and (13.13 I) , it foUows that 



h 



-L{e)\\ >5} < 2Pg{T{h) <m} 



+Pg I II — ^ L{9) II > 5/2 for some n > m 

i Y^T^ ^ ■ ^n-i 2 > s°"^^ n>m 

[2^k=i^k-i 2^k=i^k^i 

By the definition of r(/i) in fl^TD 

{m 
J2^4-i + ■■■ + 4-p) > h 
k=l 



J2(4-i + ■■■ + 4-p) > h, max^(Xj2_i + ■ ■ ■ + x]_p) < I 
.k=i 

m 

5^(4-1 + ■ ■ ■ + > h, max^(Xj2_i + ■ ■ ■ + x]_p) > I 

.fc=i 

C {m/ > /i} U Uf=i{(x2_, + ■ ■ ■ + x2„p) > /}. 

This yields 

m 

PeMh) <m}< I^rni>h) + Yl + ■ ■ ■ + ^ 

fe=l 

Consider the last term in (13.14 1) . By the inequality 

<\\ ^n^\ -L(g)|| + ||L(g)|| 



Z^A;=1 "''fc-l Z^fc=l"''A;-l 
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one has 

iiM„ii x: 



n-l 



> 6/2 for some n > m 



< P J II ^^"^ L(0)|| > ^/5j4: for some n > ml (3.16) 



+Pe -^fc — > a/^M for some n>m 



where L* = sup gg^^ II^WII- 

Combining (l37[4T) -( l3l6l yields 



supP,( ||^-L(^^)|| >5 



+P6 



I Mr,, 



< 2/(™z>;,) + 2 VsupP4||Xfc_if > /} 

-^(^)ll > -{S A V6) for some > ml 

+2Pe i LI — >-{6aV6) for some n > m\ . 

[ Ek=i4-i 2 J 

Limiting /i oo, / — » oo, m —* oo and taking into account Lemma 3.3, one 
comes to 03.12 I) . Hence Lemma 3.4. □ 

Lemma 3.5 Let Xk and T{h) be defined by ^.1 \) and ^2.2 I) . Then for any 

o 

compact set K C Ao and 5 > 

I r{h)-l \ 

hm supP, xl^^)_J 4-1 > 5 = 0. (3.17) 
Proof. In view of the inclusion 



> 5 > C {t(/i) < m} < — ^ — > 5 for some n > m 



l^k=i ^k-1 ) I Z^fc=i -^fc-i 
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one has 



+ sup Pfii < ziY^ > 5 for some n> m 

[ Efc=i 

Limiting h —>■ oo, I oo, m —>■ oo and applying Lemma 3.1 lead to (13.17 I) . 
This completes the proof of Lemma 3.5. □ 



4 Appendix. 

In this Section we cite the probabilistic result from the paper of Lai and 
Siegmund (1983) used in Section 3 and give the proof of Theorem 2.2. 

Lemma 2.2 (by Lai and Siegmund (1983)). Let {!Fn)n>o be a filtration 
on a measurable space (fi,JF), (x„)„>o and (£:„)„>o be sequences of random 
variables adapted to {J-'n)n>o- Let {P9,9 G 0) be a family of probability 
measures on (f2, J-") such that under every Pg is independent of J-'n-i for 
each n > 1. Then, for each 7 > 1/2,(5 > 0, and increasing sequence of 
positive constants c„ — 00, 

/ Ji n \ 

sup Pe I I Xi-iEi] > (5max(c„, (^^ x'^_iy) for some n > m \ — > , 



=1 i=l 



as m 00. 

Proof of Theorem 2.2. Assertion (12.20 I) easely follows from Lemma 3.12 
in [6] . For the points 6 belonging to dAp we decompose the original time series 
(11.2 I) into several components depending on the number of the roots of the 
characteristic polynomial (11.5 I) lying on the unit circle and their values. To 
this end, the characteristic polynomial (11.5 \\ is represented as 

V{z) = {z + lY'{z - lY^iz"^ - 2ZCOS0 + if'ifiz), 

where Si are either zero or 1 with Si + 62 + S3 > 1, if{z) is the polynomial of 
order r = p — Si —S2 — 2S3 which has all roots inside the unit circle. Assuming 
(without loss of generality) that r > 1, one has 

By applying the backshift operator q^^ (i.e. q~^Xn = Xn-i) one can write 
down (11.1 I) as 

q-^' {q + if' q-^^q- if 'q-^^' {q^ -2q cose/) + if 'q-''^{q)xn = En (4.1) 
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Let 9 G ri(p). Then this equation, in view of (12.19 p . takes the form 
Denote 

that is 

Introducing the vector Vn = (fn, • • • , Vn-p+i)' and the matrix 



f 1 (3, (32 ... (3p^i \ 
. 



Q 



1 1 
1 1 







yo ... 1 1 / 

one can rewrite equations (14.2 I) in the vector form 

The processes m„ and Vn satisfy the equations 

Un = -Un-1 + £n, + A^n-l + • • • + /5p-lf„-p+l = En • 

Substituting (|4X|) in ffTTl) yields 



By Theorem 3.4.2. in Chan and Wei (1988) 



n 



-2 



'i^fc-iV'fc'-i -^0, as (X). 

k=l 

Since the process Vn is stable 

lim n~~^ y Vk-iVl._i = a.s. 



k=l 
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By Donsker's theorem 

[nt] 



n 



k=i -^0 



as n ^ oo. By making use of these hmiting relations in (14.5 p . one 

trMu - - •'^ 



Kii \ Wl{s)ds, < t < 1 



where 



= (T^ < (QiQ'i) ^ >ii . 
Now by definition of t{K) in (12.2 p one has 



t{K) 



trM, 



<t)= P.(trM[,,^^, >h) = Pei^j^bl > 1) 



This and fOl imply the validity of fl^:^ for 6 G ri(p) with 

bj = l/nu . 

By a similar argument, one check (12.21 p for 6 G r2(p) with 

hi = l/{a' < {Q2Q'2)-' >ii) , 

where 



Q2 



/ 1 A /?2 ... \ 

1 -1 ... 
1 -1 ... 









1 -1 



Assume that 9 G I'sip)- Then using the equation 

q'^iq^ - 2gcos0 - l)q'P^^ip{q)xn = Sn , 
we decompose the process (11.2 p into two processes 

Un = (Un, Un-l)', Vn = . . . , Vn-p+3)' 

which obey the equations 



2C0S</) -1 \ / En 

1 
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x„,, . . . , Xn-p+iY by the following 



These processes are related with X„ — ^^„,...,^n-p 
transformation 



with 



/ 1 


Pi 






Pp-2 





\ 





1 


Pi 






Pp-2 




1 


—2 COS0 


1 















1 


—2 COS0 


1 
















1 


—2 COS0 


1 


/ 



Q3 



Further, by the same argument, one shows that 



n 



trM^nt]^tr{Q,,Q\ 



( Ht 



3; \ 



where 



that is, 



where 



4 sm (p Jq \ cos (p i 



4sin^ 



2 2 ± /r\ r\i \-l / 1 COS I 
K^.i^3J \ cosd) 1 



From here one comes to (12.21 I) with 

63 = (2 sin0)/r . 



(4.10) 



Now assume that 6 E T4{p). In this case the process (11.2 I) is decomposed 
into two vector processes 

which are defined by the formulae 

Un Xn ~\~ PlXn—1 ~l~ • • • ~l~ Pp—2-^n—p+2i "^n -^n ■^n—2 

and satisfy the equations 

Un = Un-2 + ^n, Vn + PlVn-1 + • • • + Pp-2Vn-p+2 = £n ■ 
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These processes are related to the original process fll.2 I) by the transforma- 
tion 

QaX^, (4.11) 



where 



Qa 



/I /3i ... 
1 A ... 

10-10 



\ 





y ... 1 
Further we represent Un as 

1 / Vr, 



Un=T- 

where processes ?/„ and z„ satisfy the equations 



(4.12) 



(4.13) 



T 



1 1 
1 -1 



By making use of fl4.ll I) - 04.12 I) one gets 

^ - fr(o a r 1 { ^^=1 ^^-^^'^-^ ^^=1 ^^-i^^'-i 



—2 v^" 2 —2 v^n 

^ ^ ^ l^k=iyk-iZk-i n 2^k=i^k-i 



k=l 



(4.14) 
(T-1)'. 



By Theorems 3.4.1 and 2.3 in Chan and Wei (1988), one has, as n — > oo. 



n 



k=i 

[nt] 



k=l 



[nt] 



-2 I -2 

a n 



k=l k=l 

where < t < 1. From here and fl4.ll p . it follows that 



(4.15) 
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where 



St 0\ _rj.^i[ IoWHs)ds , 



It easy to check that 

aHr{Q4Q[)-'Vt = [ {nW^{s)+r2W^{s))ds, (4.16) 
Jo 



where 



ri 



r2 



i=i j=i 



i=l j=l 



Denote 



hi = l//ii, /ii = r2/ri . 



(4.17) 



It remains to note that (14.15 Ij . fi.lG p imply 

Ve{T{h) < thVh) PeMWi, W2) > 1), as /i ^ 00 , 

where J3 is defined in (12.18 p . 

Assume that 6 G r5(p). Then equation (14.1 I) has the form 

q~^{q + l)g~^(g^ - 2gcos0 + l)q''(p{q)xn = En, r = p-3. 
Decompose x„ into three processes 
Un = q''^{q^ - 2g cos + l)g~Xg)x„ = x„ + 7ix„_i + . . . + 7p_ix„_p+i , 

Vn = q'^iq + l)g"V(g)x„ = X„ + + . . . + fp-2Xn-p+2 , 

^^n = + 1)9^^(9^ - 2gCOS0 + 1)X„ = Xn + hXn-l + t2Xn-2 + h^n-Z , 

(4.18) 

where ji,fj,tk are the coefficients of the corresponding polynomials. These 
processes satisfy the following equations 

Ufi Uji—l ~\~ £ni 

Vn = 2Vn-l COS - V„,-2 + £n, 

Wn = -(3iWn-l - ... - (3p-3Wn-p+3 + ■ 

Introducing vectors Vn = (vnyVn-i)', Wn = (w„, w„_i, . . . , w„_p+4)' and the 
matrix 



/ 1 


71 




7p-i 


1 


fi 




fv-2 





1 


h ••• 


fp-2 


1 


ti 


t2 h 








1 


tl ^2 ^3 


... 



(4.19) 



1 ti t2 / 
26 



we write down (14.18 I) as 



Un 

Vn I = Qs^n • (4.20) 



From here and fll.4 1) 

n~HrM^r.t] = tr{Q,Q',)-'C[nt]/n^ , < t < 1, (4.21) 

where 

Cn={ ELiVk-iu,-, ELiVk^iWU 
\ El=lW,.^u,.^ ELiWk-iVU ELiWk-iw;,_, 

By Theorems 3.4.1, 3.4.2 in Chan and Wei (1988), 

n-' ELi W,.,u,., ^ , (4.22) 
as n — > oo. Due to the stabihty of the process Wk, 

n 

hm V Wk-iWk_i = a.s. (4.23) 



k=l 



By the Theorem 2.3 therein 

[ni] [nt] 



k=l k=l I 

(4.24) 

where Ji and J2 are defined in (12.18 p . 

jj _ 1 f 1 COS0 \ 

4sin^0 V COS0 1 J ' 
Limiting n — 00 in (14.21 p and taking into account (14.22 P - (I4.24 p one gets 

n~'^tr M[nt] =^ m{t) as n ^ 00 , (4.25) 

where 

i{t) = aHr{Q,Q',r'(^^^ (4.26) 
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Ji{W^,t) 

J2{Wi,W2;t)H 

By easy calculation in (14.26 I) one finds 



J2(Wi,W2;t) 

m{t) = K,iiJi{W3;t) H ' {k22 + K,33 + (1^23 + K.32) COS( 

4 sm 



where Kij = < (Q5Q5) ^ >ij- 

Now by the definition of r(/i) in (12.2 \\ and (14.25 p we obtain 

Pir{h) < hVht) = P(trM,. > h) , 



trM, 



n^^hl > 1) - nm{t)hl > 1) . 



Setting 



yields 



bl = {4 sin^ 0}/ {^22 + H33 + (1^23 + 1^32) cos 0} 

,2 
5 



^2 = Kiifof 



(4.27) 



P(m(t)62 > 1) = P{J,{Wu W2, Ws; t) > 1) , 

where J4 is defined in (^JEJ), that is, fl^:^ holds for 9 e T^^p). 

One can check that for 9 G Tq{p) the limiting distribution of r(/i) coincides 
with that for 9 G r5(p) with Bq = 65. 

By a similar argument it can be shown that for 9 G ^jip) 

lim P(r(/i) < MV/I) = P{rh{t)b^j > 1) , 

ft— >oo 

where 

m{t) = knJiiWs, t) + K22 Ji(W^4; t) + 

H , ■ 2 i ('^33 + «44 + («34 + «43) COS0) , 

4sm 

= cr^ < (QrQy)"^ >iji Qj is corresponding transformation matrix which 
relates the original process with the decomposed ones. 
Setting 



6^ = (4 sin^ 0)/ (K33 + /€44 + (/t34 + K43) cos 



2 .. .2 ' (4.28) 



one comes to (12.21 \\ . This completes the proof of Theorem 2.2. □ 
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